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ABSTRACT 

A study was undertaken to, develop guidelines for the 
interpretation of the parameters of three multidimensional item 
response theory models and to determine the relationship between the 
parameters and traditional concepts of item difficulty* and 
discrimination. The three models considered were multidimensional c 
extensions bi the one-, two-, and three-parameter logistic models » 
The three models were found to have quite distinct definitions of 
difficulty and discrimination, making it imperative that the user 
take care to ensure that the characteristics of the model/used match 
the characteristics of the data to which it is applied. The 
guidelines developed as a result of this study provide information 
that should be helpful in ensuring that the appropriate model is used 
for a particular application* (Author) 
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The Definition of Difficulty and Discrimination 
for Multidimensional Item Response" Theory Models 
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Multidimensional item response theory (MIRT) models have, been available 
for at least twenty years '(Bock and Aitkin, 1981; Mulaik, 1972; Rasch, 
1961; Samejima, 1974; Sympson, 1978) f but little use has been made of these 
procedures. The reasons for- the lack of use are threefold: (a) practical 
computer programs for estimating the parameters of the models were 
unavailable, (b) examples of the application of the models were not .present 
in the literature to stimulate interest, and (c) information was not available 
on how to interpret the results of the procedures. Some gains have been - 
made in the first two of the deficient areas. Programs for the estimation 
of 'the parameters of at least one class of MIRT models are now available. 
(Bock and Aitkin, 1981; McKinley and Reckase, in press) and several 
applications of the models have appeared in the literature (Bock and Aitkin, 
1981; McDonald, 1967; McKinley, 1983; McKinley and Reckase, 1983). However, 
guidelines for the interpretation of the results of MIRT models are still 
not available in the literature* The purpose of this paper is to present 
some initial guidelines for the interpretation of the item parameters and 
to relate the parameters to the traditional measurement concepts of item 
difficulty arid discrimination. 

Three approaches will be taken in presenting the guidelines for the 
interpretation of the results of the application of the MIRT models* 
First, the form of the itettr response surface (IRS) for three different MIRT \ 
models will be presented and the effect of the model parameters on the / 
shape of the surfaces will be indicated. Second, the concepts of item \ 
difficulty and item discrimination will be defined for the MIItT- models and 
will be related to the IRS. Finally, the interpretation of the model 
parameters will be discussed and guidelines will be given for the use of 
parameter estimates. 



The Multidimensional Item Response Theory Models 

Three different MIRT models will be used as .examples in this paper to 
show the geheralissabiiity of the concepts presented. The models used are: 
(a) a multidimensional extension of the two-parameter logistic model (M2PL) 
(McKinley and Reclcase, 1982); (b) a multidimensional extension of the 
three-parameter logistic model (M3PL) (Sympson, 1976); and (e) a 
multidimensional extension of ttie Rasch- model (M1PL) (Mulaik, 1972). 
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Although these three models are generalizations of a single class of XRT 
models, they are all distinctly different models and no one of them is a 
special case of any of the others. 

The M2PL model is a special case of a general model presented by Rasch 
tl%l). The equation for the model is given by 



1 + e 



where p (X_ = 1 | a.» d,^ 8k) is the probability of a correct response to 

Item i by Person j > a is a vector of discrimination parameters for Item i, 

d- is a scalar difficulty parameter for Item i, and 6. is a vector of 

1 J 
ability parameters for Person j. Both the a.- and 0 6 .-vectors have the same 

— i ~J 

number of elements which is dependent on the dimensionality of the data 
being modelled. Note that there is only one difficulty parameter in this 
model, but a number of discrimination parameters. The model has this form 
because it was found to be impossible to estimate multiple difficulty 
parameters when using this model (McKinley and Reckase, 1982). 

The M3PL is an extension of the three-parameter logistic mod^l presented 
by Birnbaum (1968). This model is given by * 

Da,, (8., - b,_ ) 
n ik j k ik 

P(x «l!a ,b.,c ,6 ,) = c. + (1 - c.) 7T — ——7- — . (2) 

lj 1 - J 1 1 W 1+ Da ik (6 ik ~ h ik> 

1 + e J 

\ 

where b is a vector of difficulty parameters, and c i is a scalar parameter 

indicating the lower asymptote of ^the IRS. The other symbols are defined 
above . 

The M1PL model is an extension of the unidimensional Rasch model 
(Rasch, I960). This model is of the form 

P(x.. - l|e.,6j = — — ■ (31 

k»t 

where e.^ is a vector of item easiness parameters. The elements of this 
vector are of opposite s>ign to the uSual difficulty parameter. 
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The Item Response Surface Defined by the Models 

• In order to gain a better understanding of the characteristics of each 
of these three models, two plots of the item response surface defined by 
each of the models will be presented for cases limited to two dimensions. 
The plots have been produced using item parameters thac have been selected 
to accentuate the effects of the parameters on the shape of the surface- 
By contrasting the two surfaces that are being presented for each, model, an 
intuitive grasp of the effect of the parameters can be obtained. 

M2PL 

The IRS's for the M2PL model are shown in Figure I. The parameters 
used to generate these plots are given in Table 1. Notice that the surface 
in Figure la tends to rise closer to the (-3,-3) point- than the surface in 
Figure lb. This is a function of the difficulty of the two items. For an 
examinee population in which 6 X , and 6 2 are distributed N(0,1) with p Q == 0 

e l e 2 

a larger proportion of the examinees will ctbtain a correct response to Item 
la than to Item lb. This fact is reflected in the d-parameter- for the 
item. The d-parameter for Item la is much larger than that for Item lb, 
indicating that Item la is easier for the population described above. Item 
la may not be easier for every population, however. For example, for a 
population concentrated at (3,-3) on the 6 plane, Item lb is easier than 
Item la. A definition of item difficulty that addresses this issue will be 
presented later" in this paper. 

Table 1 

Item Parameters Used to Generate the Surfaces 
for the M2PL Model 



Parameter 



Item a t a^_ 



la .5 1.5 1.0 

lb 1.75 .25 -1.5** 



A second characteristic of the IRS for the M2PL model can be noticed 
in Figure 1. In Figure la, the surface increases more quickly parallel to 
Dimension 2 than to Dimension 1. For that item, a 2 is much larger than a x . 
These parameters control the steepness of the surface. For Item lb, the 
surface increases very slowly parallel to Dimension 2. For that item, a 2 
is much smaller than a x . 



M3PL 

The IRS's for the M3PL model are shown in Figure 2. The parameters 
used to generate the plots for this model are givBn in Table 2. One obvious 
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Figure 1 

Examples of Item Response Surfaces 
for the M2PL Model 




Figure lb 
Parameters 




Parameters 
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difference betweea the two plots shown in Figure 2 is that the lower asymptote 
for the two surfaces are quite different.. The lower asymptote is controlled 
by the crpardmeter , which has a value of .1 for Item 2a and .5 fir Item 2b. 
The surface cannot drop below the value of the c-parameter on the probability 
scale. < : 

% . Table 2 



Item Parameters Used to Generate the Surfaces 
for the M3PL Model 









Parameters 






Item " 


a, 


a 9 


b/ 


b, 


c 


2a 


.5 


V5 . 


-.5 


-1.5 


.1 


2b 


1.75 


.25 


0 


*-2.0* 


.5 



A second, quite obvious difference in these two plots is the general 
shape. The surface for Item 2a increases fairly quickly along both the 6 X 
and 8 2 axes , while the surface for Item 2b increases mainly along the 
6 x -axis. Tliese results are due to a combination of the effects of the a- 
and b-vectors. The a-parameters control the rate of increase of the surface 
in the area around the point defined by the b-values. For Item 2a, the 
curve increases more quickly along the 6 2 dimension than the 6 X dimension 
because a 2 is much larger than a x . The opposite is true for Item 2b. 

The IRS's for the M1PL model are showtf in Figure 3. The parameters 
used to generate the plots for this model are given in Table 3. These two 
plots almost look like mirror images of each other because of the reversal 
in the signs of the e-parameters for Item 3a and Item 3b. Although the 
.curves along the 8 X - and 8 2 -axes are of the same shape, they are shifted to 
different locations by the selection of the e-parameters. Figures 3a and 
3b are merely showing a different segment of the same surface. The point 
on the surface at (-1,.5) on Figure 3a is the same as the point on the surface 
at (1,-1.5) on Figure 3b. The two surfaces differ only by a translation. 
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Table 3 

Item Parameters Used to Generate the Surfaces 
for the M1PL Model 



4 

Parameters 



3a i.o - .5 

3b -1.0 1,5 



Definition of Item Difficulty and Discrimination 

r From an analysis of Figures 1 thru 3, it is clear that the parameters 
of the models have a direct influence 6n the shape of the IRS's. However, 
the relationship between the shape of the surface and the parameter values 
is not always simple. In or^^r to clarify "the relationship, the common 
psychometric concepts of imk difficulty and discrimination will be generalized 
to the MIRT models. The relationship between these concepts and the parameters 
will clarify the meaning of tha parameters. * 



Definition of Item Difficulty 

For unidimensional item response models, the difficulty of an item is 
defined as the point on the ability scale below the point of inflection of 
the item characteristic curve* This point calh be determined mathematically 
by solving the second derivative of the^item response function with respect 
to 9 for zero. For the MIRT models, item difficulty will be defined in the 
same way— as the root of the second derivative of the Item response functioii. 
However, in the multidimensional case the second derivative yields *a matrix 
of the form 

6 2 F 6 2 P figp .... 

60^ dQi&z 8Q x Qq • 

6 2 P 6 2 P ..... 

68 2 8 t 6e 2 2 



and a solution may not exist for which all the elements are equal to zero. 
This is equivalent* to saying that the point(s) of inflection jpay be different 
depending on the direction relative to the axis that is taken a along the 
surface. Therefore, the second directional derivative (Kaplan, 1952; p. 124) 
v, will be used to define the multidimensional difficulty of an item* For the 
two-dimensional case, the second directional derivative in direction a, 
with respect to the e^axis is given by? 




sin at cas at + sin 2 (x • (4) 

fir 2 

In most cases this definition yxeids one or mor& functions as a result, 
rather than a single value. Examples of the use of this definition will 
now be given using the three models presented above, t 

M2PL If PCX^ = 1 la., d., ) is. set equal to P for the sake of 

convenience, the second directional derivative of P with respect to the 
f-vector in the two dimensional case is equal to 

? 2p - a 2p( 2 p 2 - 3P + 1) cos 2 a + z, a P(2P 2 - 3P + 1) sin a cos a 

x \ 12 

, ' ■ ; (5) 

■ + a 2 P<2P 2 - 3P + 1) sin 2 a 

2 

) 

# 

for the M2PL model. To determine the difficulty of the item, the second 
directional derivative is set' equal to isero and solved for the appropriate 
values of P. The solution yields P = 0, .5, and 1. jfinee P « 0 and 1 are 
degenerate cases that occur when 8 ~ -*> and respectively, only P = .5', 
defines the difficulty of the item using this model. Thus, the difficulty 
of an item for the M2PL model is defined by the intersection of the IRS 
with la plane parallel to the 0 -plane at P .5* The equation for the 
intersection for the two-dimensional case of this model is given by 

V a ilV + a i2 6 2j * 0 " (6) . 

This is the equation for a straight line. The dashed line on Figures la 
and lb shows the difficulty line for the two items „shown. When a person's^ 
position in tb 8-plane is behind^that line, the probability of 3 correct 
response is greater than. 5. If it is in front of that line, the probability 
xs less than .5* ^ 

M3PL If P(X = ija b., c, 6.) is set eqiual to P and the first 

term in the product is set equal, to P x and the second term is set equal to 
P 2 , the second directional derivative of P with respect to P for the two- 
dimensional case is given by 

7 2 P * (1 - c)a 2 P P (2P 2 - 3P + 1) cos 2 « 
a 112 1 - I 



+ (1 - c)a *a P P (1 - P )(1 - P ) sin a cos a m 
. 12 12 2 1 1 1} 



+ (1 - c)d "P P (2P 2 - 3P 4-1) sin-a 




*" for the M3PL model. To determine °the difficulty of { the item, the second 
directional derivative e is set equal to zero and solved for the appropriate 
values of a,, b., and c... The solution is much more complicated for the p 

* - 

Jo 
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M3PL model than for the M2PL model. The solution for the line of inflection 
is dependent on the angle of approach to the slope relative to the 6-a'xes. 
If the line of inflection is determined parallel to the 8,-axds (a = 0°), 
the solution is given by the equation 8 2 =: b 2 . The projection of this line 
on the 8,, 8 2 plane- is a straight line parallel to th« 8-,-axis. If the 
line of' inflection is determined parallel to the 8 2 -axis*(a =90°), the Jfr 
solution is given by the equation 8 t = b^ The projection of this line on 
the B xt 6 2 plane is a straight line parallel to the 8 2 -axis. In "both 
cases, the lines of inflection divide the©!, 8 2 plane into two regions, 
one having a relatively low probability of a correct response, and the 
other having relatively high probability of a correct response,. The lines 
of inflection for* Items 2a and 2b are shown as dashed lines on Figure 2, 

M1PL The second directional derivative for the MIPL model is given 

V P « 1 1 U + e 2 2 J cos 2 a 

(8 + e ) (9 + e ) 
- 2e 1 1 e 2 2 sin a cos Q ( 8 ) 

(9 + e ) / (8 + 0 e )\ 

To determine the difficulty of an item using this model, the second directional 
derivative is set e q ual to aero and solved £or the values of a.. Different 

functions are obtained as solutions for this model depending on the angle 
of approach to the surface relative to the 8 x -axis. If the direction is 
taken as parallel to the e^axis, (a = 0°), the solution is given by the 
function, 8 2 = -e 2 . If the direction taken is paralle 7 .c the 8 2 -axis 
(a = 90°), the solution is given by the funtion 8 X * -<».. It the direction 
of solution £s halfway between the Q t and 8 2 axes (a % 45°) , M t e solution 
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e (8l + ei) + e (6 2 +e 2 ) 



(9) 



The solution for a = 0° and a = 90° are shown as dashed lines in Figured. 
Note that just as with the^ other two models, the lines of inflect ioli divide 
the Q-plane into a region with low probability of response and a region 
with a high probability of response. In this case, the low probability 
region is the quadrant nearest (-3, -3) and the high probabilitv region is 
the quadrant near (+3, +3). 
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Dgjlgjj, iQn of Item Discrimination 

In the unidiraensional IRT models, the item discrimination is related 
to the slope of the ICC at the point of inflection. In the multidimensional 
case; the discrimination can be defined iji the same way, b, ; the direction 
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of the slope relative to the 6« ,^xes must be specified* The slope may b* 
quite differeuc depending on the direction in which it is determined. The 
value 'of the slope at the line of inflection can be determined by evaluating 
the first directional derivative at the points on the line of inflection. 
The first directional derivative for the two dimensional case is given by 



&V 6V 



wher^ a is the angle with the e^axis. Generally the slope is of greater 
interest when a equals 0° or 90° than for other angles, since these cases 
indicate, the usefulness of the item for measuring ability on the.Ox and 0 2 
dimensions, respectively. Any other direction can also be used, however/ 
to determine the discriminating powefc of the item for weighted composites 
of 6i and 6 2 • 

M2PL The directional derivative for the M2PL model is given by the 
equation 

7 a P " a 1 P(1 " P) cos a + a 0 p(1 ~ p ) sin ct « (11) 

When the derivative is determined for a - 0°, the slope is given by'a t P( 1~P) 
Since 3 the line of inflection is defined by P = ,5 for this model, the slope 
in, a direction parallel to the Si-axis is equal to a x /4 all along the 
- difficulty line. This fcact shows that the discrimination of the item 
relative to 6 X is dependent on a x . Likewise when a = 90°, the slope at the 
line of inflection is a 2 /4. The a 2 parameter controls the discrimination 
of the item relative to the 8 2 -dimension, 

M3PL The directional derivative for the M3PL niodel i$ given by the 
equation x 

% 7 P = (1 - c )Da F P (1 - P ) cos a + (1 - e)Da P P (1 - P ) sin a (12) 
112 1 *2 1 2 2 

where all the symbols have been defined at Equation 7. The slope of the 
f IRS at this line of inflection when a = C° is given by (l-c)DaxP 2 /4. Thus, 
for this model, tfie slope along the line of inflection* Q^- b lt is dependent 
on 8 2 , the parameter that controls the value of P ? . The slope at the line 
of inflection, 6 2 = b 2 v , when«or = 90° is given by (l-c)Da 2 Pi/4, which is 
dependent on Q t . This means that, according to the M3PL model, the 
discriminating power of an item for a particular dimension changes depending 
on che level of ability on the Other dimensions . As the level of v ability 
on, the o # ther dimensipns increases, the discriminating power on the first 
dimension increases. For the two-dimensional case^ as ability on the 
second dimension increases, the slope parallel to 8 tl approaches (l-c)Da 1 /4. 
Similarly, the slope c at the lijie of inflection approaches . (l-c)Da 2 /4, for 0 2 
as 6j increases. 

h 
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iUPl Yftp di?cviuv.»,a de>>ivatfiw tm Uw> Ml PL rcodel 20 giaw fey the 



Q 



I13J 



V J + e 



2.0 



The h*t the IRS at the line of mfSeetaon, ^ ~e 2 , when © ^ © 

mveu w lm+e^ e s*®3 , ) 2 .. mm a * go®, t^ e slope of Else surface a Song the 
Sine @o & -e^ fi eg givesrbv i/(2+e 



Tlt^s,. for the M?PL raodel, ac with the M3PL model, the diser initiating^' 
power oi an iter? oh a parBacnlar dimension %s dependent @n the level of 
ability m the pother dimensions. However, for the MlpL esmlel, all itgias 
jre equivalent \u the it discriminating power' for a particular combination 
of * , K© discrimination parameter is present in the eorfel. Mote 

that fior the Ml PL fsodel, the poi^ik of greatest discrimination on the Q r - 

'liBeao&on e?curc when %> - *a>. At 1 that poant, the sloae m equal to J- Ac 

use va&we £©r^ @ 2 increases, th^ discriminating power for the Q % dimension 
decreases. The same result 'ean. be obtained for the ©o-dsme.nsion s 



The? definitions of difficulty and discrimination presented m. this * 
paper eao-be used w much" the bame .way that their unidisRensionai colter-, 
imttQ are-~item8 can be compared on their difficulty and items ear* be 
selected for a test based on their discrimination* However, before thme 
men ean be rnade of these characteristics ©f the items, a decision must be 
matte concerning wbith of the JO EI models to ts^e, The definitions of difficulty 
mul discrimination far these models can fee used to assast in making that 
1 <|h 2 Gfion* 



Tine definition of .difficult for ft the model 210 quite different frow 
that for' the M3PL or MlPt nodelsV .In both oa these latter cascta, there is 
c-ooeatially a separate difficulty parameter for each dimension being -measured, 
In the ea5>e ®i the model, this fact is a result of the non-compensatory 
suture of this model, • No matter how. high the ability is on one dimension, 
^raiwiot. compensate for the laefe of ability on the other dimension- The 
nWL tximel as compensatory, * and ad a result, the difficulty of the item 
(■fcymi&s on ttae'^ability on ail dimensions. 

^ : the mv%m>&#l yields two difficulty functions for' a different reason 
[Vmn Uw nWL m$M<, , The dimensions of this oiodel affect each other in an 
*sn«si<ial way, Jhe -level of ability oh one. dimension fixes the range of 
e»lf©efc:.a*r the s^.her dimensiotto for example, if ^the sum of ® fi and e g 10 § 



in 



for Dimension 1 1 tjhe minimum probability of a correct- response for the 
person on the item is ,5. The second dimension ean only determine how much ' 
above .5 the probability will fall. The surface has the same shape at that 
point as at am other po'int, but the range ®ff the probabilities has been 
•reduced- Tims, the definition of difficulty 'at ail levels of ability stays 
the same. 

The - definition of discrimination for the three models differ consi'derabi 
For the the discrimination of the iteen is defined as|t constant for 

.each .dimension. For the H3PL model, the discriminating power, of an item on 
one dimension is dependent dn the ability on the other dimensions.. For 
thia*model, the discriminating power of an item on a dimension increases as 
the- ability 0m the other dimensions increase* 3?or the ftiPL model, the 
discriminating p©yer of an item on one jiim^sxilimi is also dependent npotf the 
ability level on the other -dimensions, except for this model, the discrimi- 
nating pme$ @n a dimension declines as ability on the other dimensions 
increases * 

These three models yield clearly different definitions of discrimination 
Before any of the models is applied, the user should be sure that the 
characteristics ©f the m©dei match the diaraeteristics of the data being 
analysed* The. inforaation presented here should help in mousing that the 
appropriate model is me<A for a particular application. 
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